Systems and methods for image reconstruction

ABSTRACT

Imaging systems comprising x-ray image reconstruction systems combined with optical imaging and/or tracking systems. In some embodiments, the imaging system may comprise an x-ray image reconstruction system configured to generate three-dimensional image data of at least an internal portion of a target object under a surface of the target object and a three-dimensional optical imaging system configured to reconstruct an image of at least a portion of the surface of the target object by generating surface three-dimensional image data. The three-dimensional optical imaging system may be registered to the x-ray tomosynthesis image reconstruction system such that three-dimensional image data from the x-ray tomosynthesis image reconstruction system and three-dimensional image data from the three-dimensional optical imaging system may be used as a constraint to improve image quality of an image of the target object.

RELATED APPLICATIONS

This application is a continuation-in-part of co-pending U.S. patentapplication Ser. No. 14/198,390, filed on Mar. 5, 2014, and titled“IMAGING SYSTEMS AND RELATED APPARATUS AND METHODS”, which claims thebenefit under 35 U.S.C. §119(e) of U.S. Provisional Patent ApplicationNo. 61/773,025, filed on Mar. 5, 2013, and titled “IMAGING SYSTEM”. Thisapplication also claims the benefit under 35 U.S.C. §119(e) of U.S.Provisional Patent Application No. 62/313,041 filed Mar. 24, 2016 andtitled “SYSTEMS AND METHODS FOR IMAGE RECONSTRUCTION.” Each of theaforementioned applications is hereby incorporated by reference hereinin its entirety.

STATEMENT REGARDING GOVERNMENT SUPPORT

This invention was made with government support under SBIR grant number1456352 awarded by the National Science Foundation. The government hascertain rights in the invention.

SUMMARY

Surgeons and interventional radiologists use medical imaging to guidetheir procedures, which procedures are referred to as Image GuidedInterventions (IGI). In surgery, IGIs are most commonly performed with aC-arm.

A C-arm is an intra-operative x-ray system that creates real time 2Dprojection images. This imaging modality is called fluoroscopy. C-armsare popular because they are economical and their use does not lengthenthe procedure time.

An alternative option is to use intra-operative 3D x-ray imagers. These3D imagers include CT scanners, or Cone Beam CT (CBCT) scanners orC-arms. These systems provide 3D representations of the anatomy, whichcan be very valuable for complex anatomy or when precise 3D localizationis important (e.g., for oncology and/or spine surgery). Such 3D imagesare static, and systems need to be coupled with navigation systems tomimic real-time imaging. Navigation systems can also be used withpre-operative imaging.

However, even though these 3D imagers provide superior visualizationwhen compared to fluoroscopy, it comes with drawbacks. First, thecomplexity and time required for the procedure is lengthier. Further,intra-operative scanners enclose the patient and do not provide thesurgeon with easy access to the anatomy being imaged and CBCT C-armshave moving parts that could interfere with patients, users, and/orbystanders (surgeons and staff).

Some of the methods, apparatus, and systems disclosed herein relate tointra-operative x-ray scanning. In some embodiments, these methods andsystems can advantageously provide fast 3D reconstructions (in nearreal-time), which obviates the need for using surgical navigationsystems. In some embodiments, the systems can have an open geometry thatallows the user to access the anatomy during imaging, which can improveprocedure workflow and/or integration with other systems. Someembodiments may alternatively, or additionally, be configured so asavoid having any exposed moving parts—i.e., any exposed parts that moveduring an imaging process that may expose patients, users, and/orbystanders to injury, for example.

In some embodiments, the system may comprise: a) a gantry for moving aplurality of radiation sources through one or more paths; and b) one ormore radiation detectors, which may be configured to move or may bestationary relative to the patient and/or path(s). In some embodiments,one or more of the paths may comprise a continuous path. One or more ofthe paths may comprise, for example, a path on which a radiation sourcecontinuously moves in a single direction. Thus, a plurality of the pathsof the plurality of radiation sources may overlap, wholly or in part. Inother embodiments, one or more of the paths may be oscillating paths(i.e., the radiation source(s) oscillate along the one or more paths),and need not overlap with any of the other paths among the otherradiation sources.

In other embodiments, a single moving radiation source may be provided.With respect to such embodiments, the moving radiation source may beconfigured to move within an enclosed source gantry or other suchenclosure configured so as to avoid having any exposed moving partsduring imaging. It should be understood, however, that one or morefeatures or components of such a system may be configured to movebetween imaging sessions so as to, for example, allow for proper patientpositioning. Such systems should still be considered as being configuredto avoid having any exposed moving parts during imaging.

With respect to the source gantry, the detection device may be placed onthe opposite side/hemisphere of the source gantry with respect to thepatient. The system may further comprise c) a processor for repeatedlysampling the radiation detector(s) as the plurality of radiation sourcesmove to generate the plurality of radiation absorption images for eachradiation source; and d) a computer and computing program applying areconstruction algorithm to the radiation absorption images to generatea 3 dimensional reconstruction of the object's region of interest. Thecomputing program may be configured to update the 3 dimensionalreconstruction (or information related/related to it). The system mayfurther comprise e) a display or interface to provide the 3D datasetinformation (or information related/extracted from it) to a user.

A method can be performed to create a three-dimensional and time varyingreconstruction of a region of interest of an object. In someimplementations, the method may comprise acquiring radiation absorptionimages of the object region of interest by moving a plurality ofradiation sources through one or more paths. The radiation absorptionimages may be acquired by one or more radiation detectors. The radiationdetector(s) may be repeatedly sampled as the plurality of radiationsources move to generate a plurality of radiation absorption images foreach radiation source. The projection geometry may be repeatedlyobtained by the system (for example by using encoders and by“looking-up” previously obtained geometry calibration parameters).

An algorithm, such as a reconstruction and/or motion estimation andcorrection algorithm, may be applied to the radiation absorption imagesand associated projection geometries to generate a three-dimensionalreconstruction of the object region of interest. In someimplementations, the reconstruction algorithm may comprise an iterativereconstruction algorithm and/or a motion estimation and correctionalgorithm. The three-dimensional image may be updated as new radiationabsorption images are acquired by the radiation detector(s) and theplurality of moving radiation sources. This image, at least a portion ofthis image, and/or data derived from or related to the imagingprocessing/analysis may be displayed to a user. In some implementations,this step may comprise displaying visual information derived from thethree-dimensional reconstruction of an object region of interest on adisplay, such as a monitor.

The subject technology is illustrated, for example, according to variousaspects described below. Various examples of aspects of the subjecttechnology are described as numbered clauses (1, 2, 3, etc.) forconvenience. These are provided as examples and do not limit the subjecttechnology. It is noted that any of the dependent clauses may becombined in any combination, and placed into a respective independentclause, e.g., clause 1 or clause 5. The other clauses can be presentedin a similar manner.

1. An imaging system for providing image reconstruction data of anobject, the system comprising:

an array of at least two radiation sources configured to move along acurved path substantially in a plane; and

a detector not in the plane, the array configured such that theradiation sources emit radiation toward the detector in a sequence inwhich the emissions from each of the radiation sources occur atsubstantially the same frequency.

2. The system of Clause 1, wherein the curved path of the radiationsources is closed.3. The system of Clause 2, wherein the curved path of the radiationsources is circular or elliptical.4. The system of Clause 1, wherein the radiation sources move along thecurved path.5. The system of Clause 4, wherein the radiation sources oscillate alongthe curved path.6. The system of Clause 4, wherein the radiation sources are configuredto move along the curved path in a first direction and reverse directionto return toward their respective original locations.7. The system of Clause 5, wherein the curved path of the radiationsources comprises an open curved path.8. The system of Clause 7, wherein the radiation sources comprises fourradiation sources and each of the four radiation sources moves along aseparate open curved path that each have approximately a 90° arc.9. The system of Clause 8, wherein the separate open curved pathscollectively form a circle shape.10. The system of Clause 8, wherein the separate open curved pathscollectively form an elliptical shape.11. The system of Clause 4, further comprising at least one gantrycomponent housing the radiation sources, wherein the radiation sourcesmove within the gantry component while the gantry component remainsstationary relative to the detector.12. The system of Clause 4, further comprising at least one gantrycomponent housing the radiation sources, wherein the gantry componentmoves relative to the detector while the radiation sources remainstationary relative to the gantry component.13. An imaging system for providing image reconstruction data of anobject, the system comprising at least one radiation source that movesalong a curved path within an enclosed gantry and emits radiation towardat least one detector, the detector not being coplanar with the curvedpath, the radiation source emitting radiation at at least two regionsalong the curved path.14. The system of Clause 13, wherein the radiation source is configuredto move from a first location along the curved path to a second locationalong the curved path and reverse direction at the second location toreturn to the first location.15. The system of Clause 14, wherein the radiation source emitsradiation along at least two regions along the curved path when movingtoward the second location.16. The system of Clause 14, wherein the curved path of the radiationsource comprises an open curved path.17. The system of Clause 13, wherein the curved path of the radiationsource is closed.18. The system of Clause 17, wherein the curved path of the radiationsource is circular or elliptical.19. An imaging system for providing reconstruction image data of anobject and for allowing access to the object while imaging, the systemcomprising:

at least one radiation source configured to move along a path formed bya first curve lying substantially along a first plane and a second curvelying out of the first plane;

a radiation detector positioned and configured to receive radiationemitted from a radiation source with the object interposabletherebetween; and

a processor configured to receive radiation absorption data from thedetector and apply a reconstruction algorithm.

20. The system of Clause 19, wherein the processor comprises two or moreprocessors.21. The system of Clause 19, wherein the second curve lies substantiallyin a second plane.22. The system of Clause 19, further comprising generating a 3-D x-rayimage using the radiation absorption data.23. The system of Clause 22, wherein the 3-D x-ray image of the objectis generated as the first radiation source moves along the path.24. The system of Clause 22, further comprising a display for providinga visual representation of the 3-D x-ray image of the anatomy.25. The system of Clause 19, further comprising a second radiationsource configured to move along the path, spaced apart from the firstradiation source.26. The system of Clause 24, wherein the first and second radiationsources are positioned opposite each other along the path and move atthe same speed.27. The system of Clause 24, wherein the detector comprises first andsecond radiation detectors configured to move through a second path, thesecond path having a third curve lying substantially along a secondplane and along a fourth curve lying outside of the second plane.28. The system of Clause 19, wherein the path is generally a cylindersine wave.29. The system of Clause 19, wherein the path is generally a sphericalsine wave.30. The system of Clause 19, wherein the detector is stationary.31. The system of Clause 19, wherein the detector moves along a secondpath in a position opposite the first radiation source such thatradiation emitted from the first radiation source passes through theobject toward the detector.32. The system of Clause 19, further comprising an enclosed gantry forsupporting the first radiation source.33. The system of Clause 19, wherein the first radiation source ishoused in a generally toroidal-shaped structure.34. The system of Clause 19, wherein the first and second radiationsources are housed in separate structures.35. The system of Clause 19, wherein the first and second radiationsources are rotatable through continuously changing angles.36. The system of Clause 19, wherein the detector comprises separatefirst and second detectors.37. The system of Clause 19, wherein the processor is configured torepeatedly sample the detector.38. The system of Clause 19, wherein the path is continuous.39. The system of Clause 19, wherein the path is discontinuous and thefirst radiation source moves around only a portion of the object.40. A method for generating x-ray image data of an object, the methodcomprising:

moving a first radiation source along a path relative to the object, thepath having a first curve lying substantially along a first plane andalong a second curve lying out of the first plane; and

recording projection images of the patient from different recordingangles as the first radiation source moves along the path.

41. The method of Clause 40, wherein the second curve lies substantiallyin a second plane.42. The method of Clause 40, wherein the first radiation source movesalong a generally cylinder sine wave path.43. The method of Clause 40, further comprising moving a secondradiation source along the path and spaced apart from the firstradiation source.44. The method of Clause 40, wherein recording projection imagescomprises recording projection images at the same frequency.45. The method of Clause 44, further comprising setting the firstradiation source at a first energy level and the second radiation sourceat a second energy level.46. The method of Clause 40, further comprising further comprisingconstructing a 3-D x-ray image, by a processor, from the projectionimages, wherein constructing a 3-D x-ray image comprises constructing a3-D x-ray image from the subtraction projection images.47. The method of Clause 46, further comprising subtracting projectionimages taken from substantially the same position at different times.48. The method of Clause 46, further comprising subtracting projectionimages from substantially the same position at different energies.49. The method of Clause 40, further comprising constructing a 3-D x-rayimage, by a processor, from the projection images.50. The method of Clause 48, further comprising updating the 3-D x-rayimage as new subtraction projection images are produced.51. The method of Clause 48, wherein constructing a 3-D x-ray imagecomprises applying multi-resolution techniques to provide a first 3-Dimage of a first resolution and a subsequent image of a resolutionhigher than the first resolution.52. The method of Clause 48, further comprising displaying the 3-D x-rayimage on the display.

In an example of an imaging system according to some embodiments of theinvention, the system may comprise an x-ray tomosynthesis imagereconstruction system configured to generate three-dimensional imagedata of at least an internal portion of a target object under a surfaceof the target object and a three-dimensional optical imaging systemconfigured to reconstruct an image of at least a portion of the surfaceof the target object by generating surface three-dimensional image data.The optical imaging system may be registered to the x-ray tomosynthesisimage reconstruction system. The system may further comprise a processorconfigured to apply an image reconstruction algorithm to generate areconstructed three-dimensional image of the target object. Thereconstruction algorithm may be configured to use the three-dimensionalimage data from the x-ray tomosynthesis image reconstruction system andto use surface three-dimensional image data from the three-dimensionaloptical imaging system as a constraint, such as a density constraint orgeometric constraint, to improve image quality of the three-dimensionalimage data and reconstruct an image of the target object.

In some embodiments, the reconstruction algorithm may comprise aniterative reconstruction technique.

In some embodiments, the three-dimensional optical imaging system mayfurther be configured to reconstruct an image of at least a portion of asurface of a surgical instrument or implant by generating surfacethree-dimensional image data for the at least a portion of the surfaceof the surgical instrument or implant. Thus, the density constraint maycomprise at least in part a density profile derived from the surgicalinstrument or implant, and the reconstruction algorithm may beconfigured to apply the density profile of the surgical instrument orimplant as a constraint to improve image quality of thethree-dimensional image data.

In some embodiments, the system may be configured to apply a constraintof zero density from the surface three-dimensional image data. In somesuch embodiments, the target object may comprise a patient, and theconstraint of zero density may be applied to a region outside of the atleast a portion of the surface of the target object and outside of atleast a portion of a surface of a surgical instrument.

In some embodiments, at least a portion of the constraint may be derivedfrom an a priori, three-dimensional mass attenuation image registered tothe at least a portion of the surface of the target object via surfaceregistration.

In another example of an imaging system according to other embodiments,the system may comprise an x-ray tomosynthesis image reconstructionsystem configured to generate three-dimensional image data of a regionof interest of a target object under a surface of the target object anda three-dimensional optical imaging system configured to generatesurface three-dimensional image data of at least a portion of the targetobject. The optical imaging system may be registered to the x-raytomosynthesis image reconstruction system, and the three-dimensionaloptical imaging system may be further configured to generate surfacethree-dimensional image data of a tool to be inserted into the region ofinterest of the target object, and to generate surface three-dimensionalimage data of the tool as the tool moves relative to the surface of thetarget object. The system may further comprise a processor configured tocompile the surface three-dimensional image data of the tool over timeand derive a trajectory of the tool relative to the target object and adisplay configured to display at least a portion of the region ofinterest and to dynamically display a trajectory of the tool relative tothe region of interest.

In some embodiments, the tool may comprise a surgical instrument.

In some embodiments, the system may be configured to allow a user toselect a preferred trajectory for the surgical instrument relative tothe region of interest, and the processor may be configured todynamically calculate a variance metric between the preferred trajectoryand the trajectory.

In some embodiments, the display may be configured to display at leastone of a number corresponding with the variance metric and an imageillustrating both the trajectory and the preferred trajectory.

In some embodiments, the system may be configured to allow a user toselect a target within the region of interest of the target object, andto dynamically display a distance between the tool and the target.

In some embodiments, the imaging system may be configured to dynamicallyadjust the region of interest in response to movement of the tool.

In some embodiments, the imaging system may be configured to dynamicallydefine the region of interest so as to contain a point adjacent to adistal tip of the tool. In some such embodiments, the imaging system maybe configured to dynamically modify the display as the region ofinterest is defined by movement of the distal tip of the tool.

In an example of a four-dimensional imaging system according to someembodiments, the system may comprise an x-ray tomosynthesis imagereconstruction system configured to generate three-dimensional imagedata of at least a portion of a target object and a tracking systemconfigured to track movement of the at least a portion of the targetobject and generate motion data for a motion model based upon movementof the at least a portion of the target object. A processor configuredto generate a reconstructed three-dimensional image of the at least aportion of the target object over time comprising four-dimensional imagedata may also be provided. The reconstruction algorithm may beconfigured to use the three-dimensional image data from the x-raytomosynthesis image reconstruction system and to use motion data fromthe tracking system to generate the four-dimensional image data.

In some embodiments, the motion model may comprise use of a rigidtransformation.

In some embodiments, the tracking system may comprise athree-dimensional tracking system. In some such embodiments, thetracking system may comprise a three-dimensional imaging system. Thethree-dimensional imaging system may be configured to use the motiondata from the three-dimensional imaging system to generate movement ofthe reconstructed three-dimensional image.

In some embodiments, the x-ray tomosynthesis image reconstruction systemmay be further configured to generate motion data based upon movement ofthe at least a portion of the target object, and the imaging system maybe configured to combine the motion data from the tracking system withthe motion data from the x-ray tomosynthesis image reconstruction systemto generate movement of the reconstructed three-dimensional image.

In yet another example of an imaging system according to someembodiments, the system may comprise a three-dimensional tracking systemconfigured to generate a first data layer comprising motion data of atool or implant in motion with respect to a target object and an x-raytomosynthesis imaging system configured to obtain projective image dataof at least a portion of the target object and the tool or implant inmotion with respect to the target object. The three-dimensional trackingsystem may be registered to the x-ray tomosynthesis imaging system. Thesystem may further comprise a processor configured to generate a seconddata layer from the three-dimensional tracking system and from theprojective image data from the x-ray tomosynthesis imaging system. Insome embodiments, the processor is configured to use a reconstructionalgorithm to reconstruct the first data layer and the second data layerindividually, each data layer having different constraints, and theprocessor may be further configured to combine the first data layer withthe second data layer to generate a reconstructed three-dimensionalimage of at least a portion of the target object with the tool orimplant.

In some embodiments, the three-dimensional tracking system may beconfigured to identify the tool or implant using an a priori densityprofile, and the three-dimensional tracking system may be furtherconfigured to use a derived density profile based on the tool or implantto improve the reconstruction of the second data layer and therebyimprove the reconstruction of the three-dimensional image.

In some embodiments, the three-dimensional tracking system may comprisea three-dimensional optical imaging system configured to generate themotion data by tracking movement of the tool or implant.

In some embodiments, at least one of a shape and a color of the tool orimplant may be used to identify the tool or implant using an a prioridensity profile and a derived density profile based on the tool orimplant to improve the reconstruction of the second data layer andthereby improve the reconstruction of the three-dimensional image.

Additional features and advantages of the subject technology will be setforth in the description below, and in part will be apparent from thedescription, or may be learned by practice of the subject technology.The advantages of the subject technology will be realized and attainedby the structure particularly pointed out in the written description andembodiments hereof as well as the appended drawings.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and areintended to provide further explanation of the subject technology.

The features, structures, steps, or characteristics disclosed herein inconnection with one embodiment may be combined in any suitable manner inone or more alternative embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

The written disclosure herein describes illustrative embodiments thatare non-limiting and non-exhaustive. Reference is made to certain ofsuch illustrative embodiments that are depicted in the figures, inwhich:

FIG. 1 is a perspective view of an embodiment of an imaging system.

FIG. 2 depicts an alternative embodiment of an imaging system.

FIG. 3A is a schematic depiction of an alternative embodiment of animaging system.

FIG. 3B is a schematic depiction of another alternative embodiment of animaging system.

FIG. 4 is a schematic depiction of still another embodiment of animaging system.

FIG. 5 depicts yet another embodiment of an imaging system.

FIG. 5A is a cross-sectional view taken from line 5A-5A in FIG. 5.

FIG. 5B is a cross-sectional view taken from line 5A-5B in FIG. 5.

FIG. 6A is a schematic depiction of yet another embodiment of an imagingsystem.

FIG. 6B is a schematic depiction of still another embodiment of animaging system.

FIG. 7 depicts another embodiment of an imaging system.

FIG. 8 depicts still another embodiment of an imaging system.

FIG. 9 depicts yet another embodiment of an imaging system.

FIG. 10 is a perspective view of an embodiment of an imaging system.

FIG. 11 is a close-up view of the imaging assembly of the imaging systemof FIG. 10.

FIG. 12 is a flow chart depicting an implementation of a method forgenerating reconstruction image data of at least a portion of an object.

FIG. 13 is a flow chart depicting another implementation of a method forgenerating reconstruction image data of at least a portion of an object.

FIG. 14 is a flow chart depicting still another implementation of amethod for generating reconstruction image data of at least a portion ofan object.

FIG. 15 is a perspective view of an imaging system comprising an x-rayimaging system and an optical imaging system according to someembodiments.

FIG. 16 is a perspective view of an imaging system comprising an x-rayimaging system and a tracking system according to some embodiments.

FIG. 17 depicts an imaging system comprising an x-ray imaging system andan optical imaging system with a graph illustrating how using theinventive principles of the imaging system may allow for improving theaccuracy of a density profile associated with a region of interest of apatient.

FIGS. 18A and 18B depict, respectively, an unconstrained reconstructionand a constrained reconstruction of a spine anatomy.

FIG. 19 is a perspective view of an imaging system comprising an x-rayimaging system and an optical tracking system for tracking and imaging asurgical tool along with a region of interest according to someembodiments.

FIG. 20 is a schematic diagram illustrating the operation of an imagingsystem comprising a 3D optical system and an x-ray system according tosome embodiments and implementations.

FIG. 21 is a schematic diagram illustrating the operation of an imagingsystem comprising a tracking system and an x-ray system according tosome embodiments and implementations.

FIG. 22 is a schematic diagram illustrating the operation of an imagingsystem comprising a tracking system and an x-ray system according toother embodiments and implementations.

DETAILED DESCRIPTION

It will be readily understood that the components of the presentdisclosure, as generally described and illustrated in the drawingsherein, could be arranged and designed in a wide variety of differentconfigurations. Thus, the following more detailed description of theembodiments of the apparatus is not intended to limit the scope of thedisclosure, but is merely representative of possible embodiments of thedisclosure. In some cases, well-known structures, materials, oroperations are not shown or described in detail.

Disclosed herein are various embodiments and implementations ofapparatus, methods, and systems for providing imaging data. In someembodiments, the system can use multiple radiation sources that movesubstantially along a path or trajectory. The use of multiple radiationsources can increase the speed at which the system can acquireprojections from the full path, which can reduce acquisition time andlatency of the updates.

Various additional embodiments of apparatus, methods, and systems aredisclosed herein that relate to image reconstruction and/or imagereconstruction enhancement, such as, in some embodiments andimplementations, using tracking systems and/or cameras to enhance 3Dand/or 4D reconstructions for image guidance, some of which mayincorporate one or more elements of the x-ray imaging systems, such asthe multiple, moving radiation sources previously mentioned.

The following terms shall be defined herein as follows:

Imaged object: an object or a collection of objects being imaged by animage reconstruction system.

Mass attenuation reconstruction: a method of determination of massattenuation property of an imaged object over a volume.

Optical reconstruction: a method of determination of reflective surfaceof an imaged object.

3D x-ray image reconstruction system: a system that acquires x-rayprojection images and performs mass attenuation and/or linearattenuation reconstruction over an imaging object.

3D optical image reconstruction system: a system that acquire opticalimages and performs optical reconstruction over an imaging object.

Tracking system: a system that provides position and/or orientation ofobjects with respect to a reference frame.

In some embodiments, the radiation source(s) can move substantiallyalong a path(s) or trajectory(ies) that can be circular in a commonplane. The path(s) or trajectory(ies) can also be substantially along acylinder sine wave or saddle-shaped path, a spherical sine wave, ahyperbolic paraboloid path, or other three-dimensional paths ortrajectories. Other paths can be straight or linear along at least aportion of the extent. The path(s) can have multiple peaks and valleys,such as 2 peaks and 2 valleys (as along the brim of a saddle, forexample), 3 peaks and valleys, 4 peaks and valleys, 5 peaks and valleys,etc. Further, some embodiments can be configured such that the path(s)undulates with variable amplitude or height peaks and valleys. Thepath(s) can traverse or extend into and out of and/or at least partiallywithin a plane that passes through the object to be images. The path(s)can be curved in one or more planes. The path(s) can have a continuouscurve or bend. In some embodiments, the path(s) can be discontinuous,such as an open curved path, extend along less than an entirecircumference of a target space or object, or incompletely surround thetarget space or object. For example, an open curved path can comprise abeginning point that is separate or spaced apart from its end point,such as a 90 degree arc of a circle or ellipse. The path(s) can defineone or more corners, sharp turns, or discontinuities. Multiple separatepaths can be used for multiple separate sources and/or detectors withone or more sources and/or detectors moving along the multiple separatepaths.

In some embodiments, one or more of the paths of the one or moreradiation sources may be configured to at least substantially match oneor more of the paths of the one or more radiation detectors. In somesuch embodiments, for example, the source path or paths may have thesame shape (not necessarily the same size) as the detector path. Incertain preferred embodiments, the radiation sources are configured tomove at the same angular speed relative to the detection sources suchthat each source is positioned at a location that corresponds with alocation of the detector at a given moment in time. Thus, inembodiments, in which one path is larger than the other, for example,the source(s) and/or detector(s) on the larger path will move faster(but at the same angular or rotational speed) than the source(s) and/ordetector(s) on the smaller path.

In other embodiments, the detector(s) may be stationary relative to thepatient and/or path(s). The system can comprise two or more paths for atleast one radiation source and/or at least one radiation detector.

In some embodiments, the system may comprise one or more paths above atarget space and one or more paths below the target space for at leastone radiation source and/or at least one radiation detector.

For example, the system may comprise at least one radiation sourceand/or at least one radiation detector in a path above the target space,with at least one radiation source and/or at least one radiationdetector below the target space. In other embodiments, the system maycomprise at least one radiation source and/or at least one radiationdetector in two paths below the target space, along with another atleast one radiation source and/or at least one radiation detector in twopaths above the target space.

In some embodiments, the system can have radiation sources that rotatethereby allowing the system to work with a finite/small number ofsources and still have coverage in terms of angular density (forexample, in projections per degree), which is needed to have good imagereconstruction quality.

Further, in some embodiments, the system can use a source gantry that isseparate and on the opposite side of the patient versus the detectiondevice in order to provide the user with access to the patient'sanatomy. For example, the system can provide access for the user byapproaching the anatomy between the gantry and the detection device andalso provide compatibility with surgical tables. In other examples, thesystem can provide access for the user to the anatomy above the gantryand/or the detection device. In such embodiments, the system cancomprise a track.

The separation in two hemispheres (one for radiation sources, one fordetection device) makes the mathematical problem of solving for the 3Dimage (also called image reconstruction) ill posed. Thus, computerintensive iterative algorithms (based on iterative forward and backprojections) that use regularization (typically an a priori constrainthat helps the algorithm converge) may be used during such imagereconstruction.

Further, in some embodiments having multiple rotating detectors, thedetectors that are not observing the projected image may be used toobserve backscatter x-ray. The backscatter x-ray can be used to improvethe quality of the reconstruction, for example, by changing dynamicallythe regularization function.

FIG. 1 illustrates an embodiment of an imaging system 100 comprising agantry 110. Gantry 110 comprises a circular gantry configured to containand/or house one or more moving radiation sources. The term “gantry,” asused herein, should be understood to encompass any structural elementconfigured to position various radiation sources and/or detectors withina suitable location for imaging. Gantry 110 further comprises anenclosed gantry configured to avoid having any exposed movingparts—i.e., any exposed parts that move during an imaging process thatmay expose patients, users, and/or bystanders to injury, for example.Thus, each of the radiation sources (not shown) contained within gantry110 are configured such that no moving parts that facilitate suchmovement are exposed outside of gantry 110.

Imaging system 100 further comprises a detector 120, which may comprisea flat panel detector. Detector 120 may further comprise a stationarysingle digital detector.

Gantry 110 may house one or more radiation sources, such as x-raysources for example, that extend substantially along a path. The pathcan be any of a variety of shapes, as discussed above. FIG. 1illustrates one possible configuration in which the detector 120 ispositioned under the patient and the gantry 110 is positioned over thepatient 50. Gantry 110 may be configured itself to rotate, therebyrotating the one or more radiation sources contained therein/thereon.Alternatively, the one or more radiation sources may be configured to bemoved independently of a stationary gantry 110.

As illustrated in FIG. 1, gantry 110 may be configured to move the oneor more radiation sources in a circular or elliptical path above the bed60 on which the patient 50 lies. The circular or elliptical path may bein a single plane if desired. Further, one or more detector panels, suchas detector panel 120, may be positioned below the source(s) and thepatient 50 to detect radiation emitted therefrom. As illustrated, thedetector(s) 120 can be positioned on the bed below the patient. Inalternative embodiments, however, the detector(s) may be positionedbelow the patient on or within a separate housing, or may be positionedabove the patient, as described below.

In some embodiments, at least one of a) the gantry and b) the detectorassembly can be hollow. Having a hollow element with a relatively smallcross-section can allow the user to access the anatomy from the hollowportion of the source and/or detector by placing the hollow portionclose to the patient, thereby eliminating or at least reducing the riskof direct x-ray beam exposure to the surgeon, providing compatibilitywith lighting during the procedure, and/or otherwise making theprocedure more convenient and/or less risky.

In embodiments where the detector assembly is hollow it could be formedby a static detector (or assembled plurality of static detectors) or maycomprise of a plurality of rotating detectors corresponding to one ormore radiation sources. In some embodiments, the detector(s) may bepositioned above the plurality of radiation sources. Such embodimentsmay be of great value because they may allow the x-ray or otherradiation source to be beneath the patient and the detector above thepatient, decreasing the scattered radiation to the surgeon (scatterradiation tends to “bounce back” towards the source, such as towards asurgeon's feet).

In some embodiments where the source and detector gantries are close tothe patient and the user accesses the anatomy from a central opening ina toroidal-shaped gantry or through another hollow portion of thesystem, the source and detector shapes may have a portion offset in thecenter along the patient axis as to allow simple positioning of thesystem along the axis of the patient during a procedure. An example ofsuch a configuration is illustrated in FIG. 5, and will be discussed ingreater detail in connection therewith.

As noted above, the emitting path or trajectory could be of anyconnected shape: oval or bean-like or 8-like. This reduces thepossibility of the x-ray source to irradiate the surgeon and/or otherbystanders, which are likely to be standing under the narrowing part ofthe 8 or bean-like shape.

FIG. 2 illustrates an example of another imaging system 200. Imagingsystem 200 comprises two gantries, namely, gantries 210 a and 210 b,each of which comprises one or more radiation sources configured to movewithin a path defined by its respective gantry. In some embodiments,both gantries 210 a and 210 b comprise a plurality of moving radiationsources, such as moving x-ray sources. As mentioned above, in someembodiments, the radiation sources may be stationary relative to thegantry, in which case the gantry may be movable. Alternatively, gantrymay be configured to guide the radiation source(s) which may move withina path defined by the gantry.

One or both of gantries 210 a and 210 b may comprise radiation sourcesthat move within the full curved path (in some embodiments, circular)defined by their respective gantry. Alternatively, one or both ofgantries 210 a and 210 b may be configured such that their respectiveradiation source(s) move within a path only partially-defined by theirrespective gantry.

System 200 further comprises a detector 220 comprising a flat panelpositioned below table 60 (and below patient 50). As illustrated in FIG.2, gantries 210 a and 210 b may each be angled inwardly towards detectorpanel 220. In other words, detector panel 220 may be positioned along anaxis at least substantially parallel to the axis of patient 50, gantry210 a may be angled in a first direction with respect to such axis, andgantry 210 b may be angle in a second, opposite direction with respectto such axis. In some embodiments, one or both of gantries 210 a and 210b may comprise a dimension, such as a diameter in the case of circulargantries, that is at least substantially equal to a dimension ofdetector 220.

FIGS. 3A and 3B illustrate schematics for another embodiment of animaging systems 300A and 300B, respectively, comprising three movingradiation sources. FIG. 3A illustrates a system 300A comprising threemoving radiation sources, namely, sources A, B, and C, that each movealong a single circular path 305. Preferably each of these sources movesat at least substantially the same speed and in the same direction alongpath 305 (as illustrated by the arrows) such that the distance betweeneach source remains constant.

As also illustrated in FIG. 3A, each of the various sources (althoughthree are illustrated in the figure, any number of sources may be usedas desired) may emit x-ray or other radiation towards a detector 320,such as a digital flat panel detector or other such detector. Theintersection between a particular radiation source, a portion of apatient 50's anatomy, and the detector 320 may allow for reconstructionof a particular volume 55 of the patient anatomy. By moving the sourcesaround path 305, various projections of the anatomy of patient 50 may betaken from various directions and used to provide a three-dimensionalreconstruction of a volume of the anatomy as desired.

In the embodiment depicted in FIG. 3A, each of the various sources maybe configured to move along the same path 305 (although obviously atdifferent points along path 305 at any given moment), as indicated bythe arrows in this figure. However, various other embodiments arecontemplated. For example, as previously mentioned, a variety of othernumbers of sources may be used as desired. In fact, although at leasttwo sources is preferred for certain embodiments, other embodiments maycomprise a single radiation source, as described in greater detailbelow.

In addition, in other embodiments, each of the various radiationsources, or at least a subset of the radiation sources, may occupyseparate moving paths. For example, FIG. 3B illustrates an embodimentsimilar to FIG. 3A except that the three radiation sources (A, B, and C)oscillate along independent paths. More particularly, source Aoscillates between opposite ends of curved path 305A, source Boscillates between opposite ends of curved path 305B, and source Coscillates between opposite ends of curved path 305C, as indicated bythe respective arrows on these paths.

As also illustrated in FIG. 3B, the combined trajectories of the variouspaths 305A, 305B, and 305C at least substantially matches the shape ofthe single path 305 of the embodiment of FIG. 3A. Again, however, a widevariety of other numbers of oscillating paths may be employed for a widevariety of other numbers of radiation sources as desired. For example,two radiation sources may be employed, in which case, assuming thesources are configured to oscillate, they may oscillate betweenrespective paths defining semicircles that together define a circularpath. Of course, in some embodiments, technically the collective pathsof the various sources may not precisely touch one another for practicalreasons. However, a configuration substantially in the form depicted inFIG. 3B may be considered to comprise a plurality of individual sourcepaths substantially defining a collective, circular path even thoughthere may be small gaps between the various paths.

As those of ordinary skill will appreciate, the source path(s), whetherbeing a single path for a plurality of sources or a collective pathdefined by a plurality of paths taken by individual sources, mayalternatively comprise other shapes and/or sizes, depending upon thedesired application. Moreover, some embodiments may be configured toallow for reconfiguring one or more of the source paths in order to, forexample, accommodate differing patients and/or anatomicalstructures/features to be imaged.

However, certain preferred embodiments comprise at least a plurality ofradiation sources moving along one or more paths. Such path(s) may beclosed in some such embodiments. Having multiple sources may be usefulto increase the speed, angular coverage, and/or efficiency at whichimages, such as adsorption images, may be acquired. This may allow forreduced acquisition time and/or latency of imaging updates.

In addition, it should be understood that although certain preferredembodiments comprise curved radiation source paths, in otherembodiments, one or more of the source paths may be linear. In some suchembodiments, the collective path defined by all of the radiation sourcepaths may comprise a polygon. In some such embodiments, such a polygonmay approximate a curved path, such as a circle.

The system 300B, like system 300A, further comprises a detector 320,which may comprise a flat panel detector. The intersection between aparticular radiation source, a portion of a patient 50's anatomy, andthe detector 320 may allow for reconstruction of a particular volume 55of the patient anatomy. In addition, having the sources, which may bearranged in, on, or otherwise coupled with a gantry, separate from andon an opposite side of a patient compared to the detector may providethe user with access to the anatomy by approaching the anatomy betweenthe gantry and the detector and may also provide compatibility withsurgical tables, chairs, and the like.

The firing/detecting sequence of the various radiation sources anddetectors may also vary as desired. For example, in some embodiments,the sequence may be sequential. In other words, each source may emitradiation and then be detected by a detector sequentially to provide animage. In some such embodiments, each source that has been fired/emittedmay be detected prior to another source, such as an adjacent source,emitting radiation.

Alternatively, the sequence may be parallel. In other words, a pluralityof sources may emit radiation simultaneously, or at least substantiallysimultaneously, and then be read together by the detector.

FIG. 4 illustrates an alternative embodiment of an imaging system 400comprising moving radiation sources and moving detectors. As illustratedin this figure, two radiation sources A and B are configured to move ina curved path 405. Path 405 may define, for example, a circular or ovalshape. Radiation sources A and B may be positioned in, on, or otherwisecoupled with a gantry, as previously mentioned. Such gantry may bepositioned on a first side of a patient 50.

On a second side of patient 50 opposite from the first side, detectors420A and 420B may be positioned to move along a similar path 425. Insome embodiments, path 425 may have a similar, or identical, shapeand/or size as path 405. Detectors 420A and 420B may comprise flat paneldetectors. In some such embodiments, detectors 420A and 420B may betilted or angled inwardly relative to patient 50, which may be useful toassist in increasing the reconstructed volume of an image of ananatomical structure or feature.

In some embodiments, detectors 420A and 420B may move in the samedirection as sources A and B. Alternatively, detectors 420A and 420B maymove along path 425 in an opposite direction with respect to thedirection in which sources A and B move within path 405.

In some embodiments, the detectors 420A and 420B may be positioned in ahorizontal direction relative to a plane and/or axis of the patient 50and/or path 405. In some such embodiments, the detectors and sources maybe synchronized to allow for direct firing of radiation onto a detector.For example, the detector may be positioned such that the radiation willhit the detector at a perpendicular, or at least substantiallyperpendicular, angle with respect to the detector (assuming the detectorcomprises a panel or is otherwise flat).

Some embodiments may comprise a combination of sources and detectorsconfigured to move along the same path. For example, system 400 may alsocomprise two additional detectors 420C and 420D interspersed withsources A and B that are configured to move in path 405. Detectors 420Cand 420D may be configured to receive radiation from sources C and D,which may be configured to move in path 425 along with detectors 420Aand 420B.

As yet another alternative, in some embodiments, the detector(s) may bepositioned above the patient/anatomy and the sources may be positionedbelow the patient/anatomy. This may be useful for certain applicationsto, for example, provide less x-ray or other radiation scatter to theupper part of surgeons or bystanders.

FIG. 5 illustrates an alternative embodiment of an imaging system 500comprising a plurality of moving radiation sources and a detector.System 500 comprises a first enclosure 510 defining a path for twomoving radiation sources A and B. System 500 further comprises a secondenclosure 530 for a corresponding number of detectors 520A and 520B. Apatient 50 may be positioned in between the radiation sources and thedetectors. Although detectors 520A and 520B are depicted as being curvedand having a curvature at least substantially matching that of enclosure530, it should be understood that other embodiments are contemplated inwhich the detectors 520A and 520B comprise flat panel detectors.

As depicted in FIG. 5, enclosure 510 may be shaped to define a path forsources A and B that is non-planar. More particularly, enclosure 510 maybe configured in a “saddle” shape or otherwise may comprise a valley orother such offset region to allow for a patient to be partiallypositioned within this region. This may improve access to certainanatomical regions and/or may improve image quality.

Similarly, as also depicted in FIG. 5, the detector enclosure 530 maycomprise a similar shape oriented in an opposite direction to allow forcloser approximation of one or more sources and one or more detectors ata particular moment in time.

In some embodiments, a rail system may be positioned within one or bothof enclosures 510 and 530 so as to move sources A, B and/or detectors520A, 520B. In alternative embodiments, one or both such enclosures mayinstead comprise a shape that extends along an axis or a plane (orextends at least substantially parallel to a plane). In other words, the“valley” referenced above may be omitted. In some such embodiments,enclosure 510 may be part of a rotating gantry if desired. In someembodiments, one of the detector(s) and source(s) may be configured tomove and one may be stationary. For example, in some embodiments,sources A, B may be configured to move within one or more predefinedpaths and one or more stationary detectors may be used to receiveradiation from such source(s).

FIGS. 5A and 5B are partial cross-sectional views of examples ofstructures from system 500 that may be used to house, contain, and/orotherwise facilitate the positioning and/or movement of radiationsources and/or radiation detectors. FIG. 5A depicts a toroidal enclosure510. In some embodiments, toroidal enclosure 510 may be part of a gantryconfigured to position enclosure 510 above (or, in other embodiments,below) a patient to facilitate imaging of one or more anatomicalstructures positioned within a central opening, for example, of theenclosure 510. Although in some embodiments enclosure 510 may bepositioned at least substantially parallel to a plane along its entirelength, other embodiments may comprise a valley or saddle shape, asshown in FIG. 5.

FIG. 5B is a partial cross-sectional view of another structure orassembly 530 for housing, containing, and/or otherwise facilitating thepositioning and/or movement of one or more radiation detectors. As shownin this figure, like enclosure 510, structure 530 may also comprise anenclosure. However, enclosure 530 comprises a rectangularcross-sectional shape. It is contemplated, however, that in otherembodiments, enclosure 530 may comprise other shapes, and thestructure(s) associated with the radiation source(s) may be similar oridentical in shape and/or size to the structure(s) associated with theradiation detector(s) if desired. For example, in some embodiments, thedetector assembly 530 may comprise a saddle shape either alternativelyto, or in addition to, the gantry or assembly for the radiationsource(s).

As previously mentioned, structure 530 may be configured to house movingradiation detectors, such as detectors 520A and 520B, if desired.Alternatively, structure 530 may be configured to house one or morestationary detectors.

As also shown in FIG. 5B, in some embodiments, structure 530 may beconfigured to angle the detector(s) housed therein or otherwise coupledtherewith in a direction to further facilitate imaging. For example, inthe depicted embodiment, structure 530 is configured to angle detectors520A and 520B away from one another. This angling also directs thedetecting faces of these detectors towards enclosure 510, which allowsfor radiation from one or more sources contained therein to be directedtowards an intervening anatomical feature of interest and then towardsone or more detectors.

In some embodiments, a first radiation source and a first radiationdetector can form a first pair of devices. The system can have severalpairs of devices. In some embodiments, the pairs of devices can bepositioned and configured such that a source of a first pair and adetector of a second pair are positioned on the same side of a patient.The source and the detector can travel together along the same path, orat least along similar paths on the same side of the patient.

Each radiation source can be paired with and positioned opposite arespective radiation detector, such that each moves along the paths atcorresponding rates of speed. For example, the sources can move atsubstantially the same rate of speed. However, in other embodiments, thesource(s) may move at different rates of speed relative to thedetector(s) or, as mentioned above, one of the source(s) and detector(s)may be stationary. Preferably, however, the source(s) at least move atthe same angular speed as the detector(s).

FIGS. 6A and 6B schematically illustrate two alternative embodiments ofimaging systems configured to provide additional imaging by way ofbackscatter imaging. System 600A comprises two radiation sources A and Band a single, flat panel detector 620A. As shown in FIG. 6A, a portion55 of a patient's anatomy may be reconstructed by way of a transmissionimage 622A from source A and a backscatter image 624A also from sourceA. Backscatter images may be used to improve image reconstructionquality.

In some embodiments, the detector, such as detector 620A, may comprisean x-ray grid configured to only allow for x-ray transmissiontherethrough at one or more particular angles. This may be useful tofilter scatter radiation from a transmission (or vice versa).

FIG. 6B illustrates an alternative embodiment of an imaging system 600Bconfigured to provide both transmission and backscatter imaging.However, system 600B differs from system 600A in that it comprises twoseparate detectors, detector 620B and detector 620B′. Detectors 620B and620B′ are angled inwardly towards one another so as to face radiationsources A and B to facilitate imaging. In some embodiments, aspreviously discussed, detectors 620B and 620B′ may be configured to movealong with sources A and B. In other embodiments, detectors 620B and620B′ may be stationary.

At the moment of imaging depicted in FIG. 6B, a backscatter image ofregion 55 from source A is being received on detector 620B′ and atransmission image of region 55 is being received on detector 620B.However, it should be understood that at other points during theoperation of system 600B, detector 620B′ may be receiving a transmissionimage and detector 620B may be receiving a backscatter image, dependingupon the positioning/movement of the various sources and/or detectorsduring operation. It should also be understood that any number ofradiation sources may be provided as desired. However, for certainembodiments comprising more than one radiation source, a sequentialfiring sequence may be needed.

FIG. 7 illustrates another embodiment of an imaging system 700. Imagingsystem 700 comprises four radiation sources and four detector panels.However, only two radiation sources and two corresponding detectorpanels are visible in the figure. More particularly, radiation sources Aand B, which may be positioned above a prone patient 50, are shown inthe figure. Radiation sources A and B may be configured to move in oneor more paths above the patient 50 (on table 60) in order to provide animage of a portion 55 of an anatomical region of interest, such as aportion of a patient's spine for example. Two other radiation sources(not shown in FIG. 7) may similarly be configured to move about in thesame or distinct paths in order to increase the imaging speed.

Two detector panels, namely, panels 720A and 720B, may also be providedbelow patient 50. In FIG. 7, detector panel 720A is receiving radiationfrom source A and detector panel 720B is receiving radiation from sourceB. Panels 720A and 720B are configured to be moved in one or more pathson one or more tracks 730. In the depicted embodiment, a single track isprovided. However, other embodiments are contemplated in which multipletracks may be provided. Also, although not shown in FIG. 7, twoadditional detector panels may be provided if desired. As shown in thefigure, the various detector panels are angled inwardly towards oneanother so that they face towards a respective radiation source.

FIG. 8 illustrates yet another embodiment of an imaging system 800.

Imaging system 800 is similar to imaging system 700 except thatradiation sources A and B are positioned below a prone patient 50 anddetector panels 820A and 820B, are positioned above patient 50. Likeimaging system 700, imaging system 800 comprises one or more tracks 830configured to move the various detector panels in one or more desiredpaths.

FIG. 9 illustrates still another embodiment of an imaging system 900.Imaging system 900 comprises paths in which both radiation sources anddetectors move together. For example, in some embodiments, the radiationsources and detectors may be coupled together in pairs. For example, afirst source A is coupled with a first detector panel 920A and a secondsource B is coupled with a second detector panel 920B. The first paircomprising source A and detector panel 920A may be coupled with a firsttrack system 930A and the second pair comprising source B and detectorpanel 920B may be coupled with a second track system 930B. Track 930Amay be configured to move source A and detector panel 920A in a path,such as a circular or other curved path, for example, above patient 50.Track 930B may similarly be configured to move source B and detectorpanel 920B in a second path below patient 50.

FIG. 9 can be representative of two alternative embodiments of imagingsystem 900. In a first such embodiment, as discussed about, thesource(s) may be coupled directly with detector(s) immediately adjacentto one another. In a second such embodiment, the source(s) may be spacedapart from the detector(s) but in the same path (similar to theembodiment depicted in FIG. 4). With respect to the latter of these twopossible embodiments, FIG. 9 may represent two overlapping images takenat two different points in time during an imaging process within whichsources A and B, and detectors 920A and 920B, are moving.

Of course, those of ordinary skill in the art will appreciate that awide variety of alternatives are possible. For example, a greater numberof source/detector pairs may be used. In some embodiments, two suchpairs may be provided in a first path and two such pairs may be providedin a second path separated from the first path. In certain preferredembodiments, the two paths may be positioned such that a patient, or atleast a portion of a patient to be imaged, may be positioned in betweenthe two paths. In other embodiments, four source/detector pairs may beprovided in the first path and four in the second path. Preferably, eachsource/detector pair has a corresponding source detector pair in adistinct path that can be considered “linked” in some way. For example,one source/detector pair may be positioned to face a secondsource/detector pair such that radiation from a source from one suchpair will always be detected by a detector from the “linked”source/detector pair. As such, the linked source/detector pairs may beconfigured to move at at least substantially the same angular speed andmay be moved and angled so as to maintain a suitable angling to providefor such a result.

The gantries and track systems disclosed herein may, in someembodiments, be combined such that radiation sources and/or detectorsmay be moved in a rotating gantry comprising a track configured to movethe sources and/or detectors in one or more predefined paths. Forexample, in some embodiments, a chain powered by a motor may be used tomove sources and/or detectors in one or more predefined paths, such as asingle circular, oval, or other curved path.

FIG. 10 illustrates another embodiment of an imaging system 1000.Imaging system 1000 comprises an imaging assembly 1005 comprising agantry 1010 and a detector 1020. Gantry 1010 comprises one or moreradiation sources. In some embodiments, gantry 1010 may be configured tomove such radiation source(s) in one or more predefined paths. Forexample, in some embodiments, a track system may be provided, asdiscussed above. Gantry 1010 may further comprise a generator and/orbattery if desired. The battery may be embedded inside the gantryhousing to decrease the cabling between static and moving parts of thesystem. In the configuration depicted in FIG. 10, a surgeon and/or robotcan operate and/or manipulate a patient from the center of the “halo” ordonut hole of gantry 1010.

System 1000 further comprises a positioning arm 1015 coupled to gantry1010. Positioning arm 1015 comprises a C-shape that may be configured tohold gantry 1010 and/or a detector, such as detector 1020, rigid withrespect to each other. Although other shapes are possible, providing a Cshape may allow for rotation of the radiation source(s) and detector(s)together as a single unit, which may be useful to access a patient'sanatomy from different angles and/or to capture images from differentangles. However, other embodiments are contemplated in which the gantryand/or radiation sources may be positioned/moved (between imagingsessions) independently of the detector(s).

In the depicted embodiment, detector 1020 comprises a curved detector.This detector may therefore be also used as a bed or resting tray suchthat a patient may, for example, lie down or otherwise rest ananatomical region of interest on the detector panel. In alternativeembodiments, however, one or more radiation detectors may be positionedunderneath such a bed/tray/panel.

In some embodiments, detector 1020 may comprise a digital flat paneldetector configured to capture and digitize x-ray or otherelectromagnetic radiation absorption images from a conic x-rayprojection delivered from one or more radiation sources. The detector(s)and/or detector assembly could alternatively be flat or v-shaped ifdesired.

System 1000 further comprises a pair of structural raisers 1045 that maybe configured to allow imaging assembly 1005 to be moved up and down toaccommodate different table heights, patient sizes, etc.

A base 1050 may be provided to, for example, contain power supplies,counterweights, electronics, etc. Wheels 1052 may also be provided toallow for imaging assembly 1005 to be moved about.

In some embodiments, base 1050 may be configured to fit and be storedwithin a recess of a corresponding workstation comprising, for example,a computer and/or monitor. For example, in the depicted embodiment, aworkstation 1060 is provided comprising a recess 1062 for receiving atleast a portion of base 1050. Workstation 1060 comprises a monitor 1064and a computer 1066, which may be used for visualization and imagereconstruction.

FIG. 11 depicts imaging assembly 1005 of imaging system 1000 in arotated configuration. One or more portions of imaging assembly 1005 maytherefore be configured to allow for rotation to accommodate patientimaging or otherwise make the imaging process more convenient. As shownby arrow 1002, in some embodiments this may be accomplished by insertingpositioning arm 1015, which may be coupled to one or both of gantry 1010and detector 1020, into a corresponding curved housing 1017 in imagingassembly 1005. Detector 1020 may similarly be configured to move along atrack defined by housing 1017. One or more of element(s) 1045 may becoupled with one or more housing 1017 elements if desired.

Preferably, gantry 1010 and detector 1020 are movable together as a unitsuch that the relative positions of the radiation source(s) anddetector(s) are preserved. However, alternative embodiments arecontemplated in which gantry 1010, or another structure housing orotherwise containing one or more radiation sources, may bepositioned/moved in between imaging sessions independently of one ormore corresponding radiation detectors.

In one or more of the embodiments described above, the radiation sourcesmay be configured to rotate or otherwise move about a center point of acircular or otherwise curved path and move along the path. Inembodiments configured to oscillate about such a path, each source maybe configured to move from an initial or first location along the pathand then reverse direction at a second location to return to the firstlocation. As the source(s) move, they may be configured to emitradiation at at least two positions along the path. Further, each sourcecan move along a separate open curved path if desired. The open curvedpaths of the sources can collectively form a circular, elliptical, orother shape. The circular, elliptical, or other shape can be planar orlie partially or entirely out of a single plane.

For example, in some embodiments, in imaging system may comprise fourradiation sources and each of the four sources may be configured to movealong open curved paths that each have about a 90 degree arc, such thatcollectively, the four sources have 360 degree coverage (whether thecollective path is circular, elliptical, or otherwise).

FIGS. 12, 13, and 14 depict, respectively, implementations of imagingmethods 1200, 1300, and 1400 that may be performed by one or more of theimaging systems and/or apparatus discussed herein.

In any of the methods disclosed herein, “Projections” may comprise aseries of absorption projection images, each associated with thenecessary geometric parameters that describe the geometric relationshipbetween the imaged volume and the associated projections.

An example of this methodology is described in Cone-Beam ReprojectionUsing Projection-Matrices, published in IEEE TRANSACTIONS ON MEDICALIMAGING, VOL. 22, NO. 10, OCTOBER 2003. This paper is herebyincorporated herein by reference in its entirety.

In these exemplary methods, the output 3D volume may be a volumetricrepresentation that correlates to the volumetric densities of the imagedvolume. The output 3D volume can be visualized in different ways thatare relevant to the user. A typical visualization method is to show aseries of slices of the output 3D volume along certain axis, for exampleto provide coronal slices, sagittal slices, or axial slices like inComputer Tomography (CT).

In method 1200, a certain number of Projections 1201 may be obtainedfrom an imaging system, for example, the imaging systems and/orapparatus discussed herein. At step 1202, a 3D volume 1203 may bereconstructed from the imaged volume's Projections. For example, aniterative algorithm like an Algebraic Reconstruction Technique (alsoknown as ART, ref. 2) can be used. Examples of such techniques can befound in Algebraic reconstruction techniques (ART) for three-dimensionalelectron microscopy and x-ray photography, published in Journal ofTheoretical Biology 29 (3): 471-81 (December 1970). This paper is alsohereby incorporated herein by reference in its entirety.

The quality and speed of the iterative reconstruction depends on thesparsity or density characteristics of the imaged volume. In method1200, the acquired Projections may be characterized by being dense. Inorder to obtain a 3D volume with meaningful clinical information, a highnumber of Projections and/or iterations may be needed, resulting inincreased system latency. A solution to achieve faster reconstruction(and thus visualization) based on sparse Projections is described inmethod 1300, represented in FIG. 13. Similar methods that exploit thesparsity of the data have been proposed, such as in Accurate imagereconstruction from few-views and limited-angle data in divergent beamct, published in J X-Ray Sci. Technology, 14: 119-139 (2006), which ishereby incorporated by reference in its entirety.

In method 1300, at step 1301, a certain number of reference Projectionsmay be obtained.

At step 1302, a certain number of updated Projections may be obtainedusing an imaging system, for example one of the imaging systems and/orapparatus discussed herein.

At step 1303, a sparse Projection set may be obtained from the referenceand updated Projections. This could be possibly implemented using asimple subtraction between reference Projections and updatedProjections. The creation of the sparse Projections can be called aforeground extraction.

In some implementations, the reference Projections may be taken from (orbe derived from) the physical systems and/or apparatus discussed hereinor derived from the reference 3D volume 1305 by, for example,mathematical projection.

Step 1304 may comprise reconstructing the 3D volume of the extractedforeground, and may, in certain implementations, operate in a similarmanner as step 1202 in method 1200. Due to the sparsity of the extractedforeground Projections, the reconstruction algorithm requires a lowernumber of Projections and/or iterations, hence reducing latency.

At step 1306, the 3D volume of the extracted foreground may berecombined with the reference 3D volume 1305 to produce the Updated 3Dvolume 1307 that can be visualized.

The reference 3D volume in 1305 represents the imaged volume associatedwith the Projections of 1301. Reference 3D volume may be obtained, forexample, using a pre-operative CT-scan, another a-priori image, or thereconstruction of an initial higher resolution tomosyntheticreconstruction.

In some implementations, motion estimation and correction may be used tohave a reference 3D volume that best matches reference Projectionsand/or to ensure sparsity of the foreground extraction. For example,method 1400 may be used to update the reference 3D volume.

Method 1400 may be used for generating an updated 3D volume forvisualization or as means to provide a better reference 3D volume inmethod 1300.

In method 1400, at step 1401, a certain reference 3D volume may beobtained. This reference 3D volume may be obtained, for example, using apre-operative CT-scan, another a-priori image, or the reconstruction ofan initial higher resolution tomosynthetic reconstruction.

At step 1402, a certain number of updated Projections may be obtainedusing an imaging system, for example, any of the imaging systems and/orapparatus discussed herein.

At step 1403, motion may be estimated and corrected using, for examplean iterative gradient descent algorithm, resulting in an updated 3Dvolume 1404. The motion correction could be, for example, modeled basedon 6 degrees of freedom to describe translational and rotationalchanges.

Methods 1200, 1300, and 1400 may rely on obtaining a certain number ofProjections. As such, the system latency in certain implementations maydepend on the time it takes to acquire the Projections and the time ittakes to execute the reconstruction method and obtain the 3D volume.

Each of the depicted methods 1200, 1300, and 1400 may therefore be usedsequentially to provide a sequence of 3D volumes, thereby allowing theuser to visualize changes of the imaged volume.

Each of the depicted methods 1200, 1300, and 1400 may also be used in aparallel computational pipeline to provide a faster sequence of 3Dvolumes. Each reconstruction may be based on a certain number ofProjections (for example, 90), with each new execution of the methodstarting after a fewer number of Projections has been obtained from thesystem (for example 12, which is smaller than 90). In this case,multiple instances of the method may be run in parallel and the latencymay be reduced.

Each of the depicted methods 1200, 1300, and 1400 may be implementedaround an iterative algorithm (iterative reconstruction algorithms 1202or 1304, or iterative motion estimation 1403). Each method can thereforebe used continuously by updating the iterative algorithm's input as newinput becomes available.

In some implementations, one or more of the depicted methods 1200, 1300,and 1400 may be implemented as a computer program and implemented onhighly parallelized architectures, for example on General PurposeGraphical Processing Units (GPGPU).

A computer program implementing any of methods 1200, 1300, and 1400 mayuse optional multi-resolution techniques to update the volume quicklyand refine the image later (start with a low number of updated images,low projective image resolution, low number of voxels and then refinewith more images, higher resolution projective images and higher numberof reconstructed voxels).

One or more systems disclosed herein may have unique potential toexploit dual/multi energy schemes since radiation sources could be setat different energy levels (kV, or eV). For example, a plurality ofradiation sources can be used that have variable or steady energy levelsthat are generally the same or different from each other.

Some embodiments may also, or alternatively, have a unique potential toexploit digital subtraction schemes since radiation sources can quicklyoverlap each other and projection images taken from the same positionbut at different times as the radiation source(s) and/or gantry rotatescan be subtracted. Subtracted projection images can feed the 3Dalgorithm obtaining subtracted 3-D datasets. Subtracting the imageprojections may improve the quality of the reconstruction since thealgorithm attempts to reconstruct a sparser volume.

In some embodiments and implementations, the subtraction can be fromprojective images taken at different energy levels (kV or eV).

In some embodiments, improved access for surgeons and interventionistsmay be interchanged with improved access to robots performing theintervention or simplify the integration with other devices (for examplewith radiotherapy systems that target tumors).

As noted above, the path(s) of the source(s) and/or detector(s) can beused for source(s) and/or detector(s) that are positioned on a firsthemisphere of an object. Further, in embodiments in which source(s)and/or detector(s) in a second hemisphere of the object move relative tothe object, those source(s) and/or detector(s) in the second hemispheremay also move along any of the variety of paths discussed herein.Additionally, a first path in a first hemisphere may be the same shapeas a second path in a second hemisphere, a different shape, translated,rotated, mirror, or otherwise be positioned similarly or dissimilarlyrelative to the second path, as desired.

The following additional concepts are disclosed herein, which may beuseful in performing various implementations of methods and/or creatingvarious embodiments of systems embodying and/or implementing one or moreof the inventive concepts below:

Aid to a 3D reconstruction: A 3D model of an imaged object obtained froman optical camera (a 3D optical image reconstruction system) can be usedto aid the reconstruction of the imaged object by a 3D x-ray imagereconstruction system. For example, when the x-ray imaging systemcomprises an x-ray CT system, Cone Beam CT system, or a Tomosynthesissystem, such as the system disclosed in U.S. patent application Ser. No.14/198,390 titled “IMAGING SYSTEMS AND RELATED APPARATUS AND METHODS,”which application is incorporated herein by reference in its entirety.

An example of a system for imaging reconstruction using both x-raytomosynthesis and optical reconstruction is depicted in FIG. 15 at 1500.Imaging system 1500 comprises an x-ray tomosynthesis imagereconstruction system comprising a gantry 1510 and a detector 1520.Gantry 1510 comprises one or more (preferably a plurality) of x-rayradiation sources (not shown in FIG. 15). Gantry 1510 may be configuredto move such radiation source(s) in one or more predefined paths. Forexample, in some embodiments, a track system may be provided, aspreviously described.

As also previously described, imaging system 1500 comprises a detector1520 positioned on an opposite side of gantry 1510 so that at least partof a patient 50 can be positioned in between gantry 1510 and detector1520. Gantry 1510 is configured to enclose the plurality of x-rayradiation sources within an enclosed portion of the gantry. Gantry 1510is further configured so as to avoid having any exposed moving partsduring an imaging process using imaging system 1500, and is configuredto enclose the plurality of x-ray radiation sources without fullyenclosing the patient 50, or any part of patient 50, so as to allowaccess to patient 50 during the imaging process. Of course, patient 50may be replaced with another three-dimensional object in alternativeembodiments and implementations.

In addition, unlike embodiments described in connection with previousfigures, system 1500 further comprises a three-dimensional opticalimaging system configured to reconstruct an image of at least a portionof a surface of a target object, such as patient 50, by generatingsurface three-dimensional image data. The three-dimensional opticalimaging system is preferably registered to the x-ray tomosynthesis imagereconstruction system so that data from both systems can be used toimprove image reconstruction. The three-dimensional optical imagingsystem comprises one or more optical cameras configured to generatedistance/depth data for a surface of a three-dimensional object, such asRGB-D cameras 1550. The depicted embodiment comprises four such optical,depth detecting cameras 1550, two of which are coupled to the detector1520 and two of which are coupled to the gantry 1510. However, afterhaving received the benefit of this disclosure, those of ordinary skillin the art will appreciate that alternative types of optical cameras,numbers of optical cameras, and placement of optical cameras may beprovided.

Cameras 1550 may be configured to reconstruct an outline of a surface ofpatient 50, or at least a portion of a surface of patient 50, and may beused to generate one or more density constraint profiles to improve thereconstruction of a three-dimensional image of a target region ofpatient 50 or another three-dimensional object. In preferredembodiments, the three-dimensional optical imaging system is registeredto the x-ray tomosynthesis image reconstruction system. For example, theoutline or patient surface can be referenced to the same reference frameas the tomographic reconstruction.

In some embodiments and implementations, multiple separate objects maybe imaged using the three-dimensional optical imaging system. Thus, forexample, a surgical tool 20 and/or an implant, or a combination oftools/implants, may be surface/depth imaged using the three-dimensionaloptical imaging system. In the example of FIG. 15, the cameras 1550 arerigidly attached to the system 1500 and therefore may be registered tothe x-ray tomosynthesis image reconstruction system via a calibrationstep. The registration can be described or decomposed as a translation3D vector and three rotations for each camera. In some embodiments andimplementations, multiple cameras can jointly provide the patient, tool,and/or implant outline, or parts of it. However, alternative embodimentsand implementations are contemplated in which the camera(s) need not berigidly attached to a component of the system, as explained below.

As also shown in FIG. 15, system 1500 may further comprise a monitor oranother suitable display 1564, for reproducing, in some embodiments andimplementations in real or near real time, the reconstructed image.

One or more such systems, such as system 1500, may perform a volumetricmass and/or linear attenuation reconstruction. If such systems use aniterative reconstruction algorithm or equivalent, the algorithm can beconstrained with the 3D model obtained from one or more optical cameras.Such constraint could be as simple as describing the surface of theobject.

Other less simple density constraints could be used as well. The outsideof the object can be modeled with a low density (typically air) and thisvolumetric density constraint may improve the reconstruction, forexample, by reducing artifacts related to otherwise incomplete/lesscomplete data for reconstruction. Incomplete data can be limited angledata for reconstruction or limited view of a region of interestreconstruction. Moreover, the inside of the object can be modeled as acontinuous function that ties the density outside of the object with themass and/or linear attenuation from the solving model, for example fromthe 3D model in an iterative reconstruction algorithm.

Aid to a 4D reconstruction with a tracking system I: Some embodiments ofthe invention may allow for accounting for motion of an imaged object(or objects) to improve its (their) volumetric mass and/or linearattenuation reconstruction (for example, with a reconstruction based onx-rays). A method to reconstruct 4D scenes may rely on an evolutionmodel that is updated from time to time. The initial reconstruction andthe updated reconstructions may be distinguished as a typical case, butthis could be imagined in more general terms. Positional changes canhappen and be captured by a tracking system any time from the initialreconstruction to the last reconstruction (including any time inbetween). These changes may include patient/table motion, gantrydisplacements, and/or surgical tools that are the object of the massattenuation reconstruction at least partially in the field of view,separately or jointly reconstructed and tracked. In this context,tracking systems can be, for example, optical tracking systems (such assurgical navigation tracking systems), optical 3D reconstructionsystems, or electromagnetic tracking systems to name a few.

Some systems may implement an algorithm that uses x-ray imaging toobserve motions and evolves the 4D mass attenuation reconstruction basedon such observed motion. Such observations may be further improved ifthey were replaced (or combined, see next section) with motions observedfrom the other tracking system. Therefore, some embodiments and/orimplementations of the invention may deal with accounting for motionthat can be captured externally to the mass attenuation reconstructionsystem by means of tracking, for example via video monitoring of thescene. Extracted motion parameters can be transferred to the massattenuation reconstruction engine.

Thus, another example of an imaging system is depicted in FIG. 16 at1600. System 1600, like system 1500, system 1600 comprises an x-raytomographic system comprising a detector 1620 positioned on an oppositeside of a gantry 1610 so that at least part of a patient 50 can bepositioned in between gantry 1610 and detector 1620. Gantry 1610 is,again, configured to enclose a plurality of x-ray radiation sourceswithin an enclosed portion of the gantry. Gantry 1610 is furtherconfigured so as to avoid having any exposed moving parts during animaging process using imaging system 1600, and is configured to enclosethe plurality of x-ray radiation sources without fully enclosing thepatient 50, or any part of patient 50, so as to allow access to patient50 during the imaging process.

In addition, system 1600 further comprises a three-dimensional motiontracking system 1650 comprising one or more tracking cameras 1655, suchas infrared tracking cameras, and one or more markers, such asfiducials. In the depicted embodiment, three-dimensional motion trackingsystem 1650 comprises two infrared tracking cameras 1655A and 1655B,both of which are mounted on a movable stand or assembly. In addition,three markers are used to track fiducial markers that reflect infraredlight, namely, a first marker 1651 positioned on a part of the x-raytomographic system, such as on detector 1620, a second marker 1652positioned on a surgical tool 20, and a third marker 1653 positioned ona desired portion of patient 50, such as within a region of interest ofpatient 50.

The three-dimensional motion tracking system 1650 is configured toprovide motion information, such as, for example, motion absolute to thetracking cameras 1655A and/or 1655B or a fixed portion of the relatedstand/assembly, and/or motion relative between each tracked object. Suchmotion information can be used to improve a 4D reconstruction,particularly if the reconstruction is a model-based reconstructionincluding motion.

In some embodiments, the combined system 1600 may also be configured toprovide information to identify a particular surgical tool 20 that is inthe region of interest (and thus which radiodensity is expected) and/orwhere the tool 20 is located (and thus which radiodensity is expected inspecific areas of the reconstructed image). In some embodiments andimplementations, such information may be used improve the reconstructionby adding this information as a constraint to an iterativereconstruction algorithm. As previously mentioned, the tracker/camerasare shown mounted to a movable pole in the embodiment of FIG. 16, andare therefore not rigidly attached to the x-ray tomographic system.Thus, a registration between the x-ray tomographic system, thethree-dimensional motion tracking system 1650, and the surgical tool(s)20 may be performed. Because this process would be simpler (andconstant) if the camera(s) were instead fixed to the x-ray tomographicsystem, alternative embodiments are contemplated in which thethree-dimensional motion tracking system 1650 may not be movablerelative to the x-ray tomographic system.

As shown in FIG. 16, system 1600 may further comprise a monitor oranother suitable display 1664, for reproducing, in some embodiments andimplementations in real or near real time, a reconstructed image.

Yet another example of an imaging system 1700 is depicted in FIG. 17. Asshown in this image, system 1700 again comprises an x-ray tomographicsystem comprising a detector 1720 positioned on an opposite side of agantry 1710 so that at least part of a patient 50 can be positioned inbetween gantry 1710 and detector 1720. Gantry 1710 is, again, configuredto enclose one or more (preferably a plurality) of x-ray radiationsources within an enclosed portion of the gantry 1710. Gantry 1710 isfurther configured so as to avoid having any exposed moving parts duringan imaging process using imaging system 1700, and is configured toenclose the plurality of x-ray radiation sources without fully enclosingthe patient 50, or any part of patient 50, so as to allow access topatient 50 during the imaging process.

System 1700 further comprises a three-dimensional optical imaging systemconfigured to reconstruct an image of at least a portion of a surface ofa target object, such as patient 50, by generating surfacethree-dimensional image data. The three-dimensional optical imagingsystem is preferably registered to the x-ray tomosynthesis imagereconstruction system so that data from both systems can be used toimprove image reconstruction. The three-dimensional optical imagingsystem comprises one or more optical cameras configured to generatedistance/depth data for a surface of a three-dimensional object, such asRGB-D cameras 1750. The depicted embodiment comprises two such opticalcameras 1750, one of which is coupled to the detector 1720 and one ofwhich is coupled to the gantry 1710. In the depicted embodiment, cameras1750 are mounted to their respective components of the x-ray tomographicsystem using mounting posts 1752. However, again, alternative types ofoptical cameras, numbers of optical cameras, and placement of opticalcameras may be provided as desired.

Cameras 1750 may be configured to reconstruct an outline of a surface ofpatient 50, or at least a portion of a surface of patient 50, and may beused to generate one or more density constraint profiles to improve thereconstruction of a three-dimensional image of a target region ofpatient 50 or another three-dimensional object. In preferredembodiments, the three-dimensional optical imaging system is registeredto the x-ray tomosynthesis image reconstruction system. For example, theoutline or patient surface can be referenced to the same reference frameas the tomographic reconstruction. This information from both systemscan be combined to improve image resolution.

More particularly, as illustrated in the density profiles in the chartincluded in FIG. 17 (density profiles are shown in one dimension forsimplicity), using a density constraint allows the reconstructionalgorithm to find a solution that is closer to the actual density of theregion of interest 55 of patient 50. Thus, by using a three-dimensionaloptical imaging system to apply a density constraint to the surface ofpatient 50, as shown in line DC in FIG. 17, a reconstructed densityusing this density constraint (shown at line R1) may be improved upon byusing the density constraint, as shown at line R2), which is much closerto the actual density profile (shown at line AD) of the patient 50 andthe region of interest 55.

Thus, having RGB-D cameras 1750, or other suitable elements of anoptical imaging system, provide an outline of the patient 50 that can beused to constrain the solution in an iterative reconstruction algorithm,higher resolution may be provided. In other words, the density matchesmore closely the actual density. FIGS. 18A and 18B depict, respectively,reconstructions of a particular anatomical region of interest, andextending beyond the region of interest, without and with use of thisdensity constraint methodology. In the case of the constrainedreconstruction (FIG. 18B), the density beyond the surface of the patientis constrained to zero in an iterative reconstruction scheme. Thequality of the tomosynthesis reconstruction is improved, and thereduction of truncation artifacts is noticeable by comparing theseimages.

FIG. 19 depicts still another example of an imaging system 1900. Imagingsystem 1900 again comprises an x-ray tomosynthesis image reconstructionsystem comprising a gantry 1910 and a detector 1920. As previouslymentioned, gantry 1910 comprises one or more (in some embodiments, aplurality) of x-ray radiation sources and may be configured to move suchradiation source(s) in one or more predefined paths, such as along acircular path adjacent to a perimeter of gantry 1910, for example.

As also previously described, imaging system 1900 also comprises athree-dimensional optical imaging system configured to reconstruct animage of at least a portion of a surface of a target object, such aspatient 50, by generating surface three-dimensional image data. Thethree-dimensional optical imaging system is preferably registered to thex-ray tomosynthesis image reconstruction system so that data from bothsystems can be used to improve image reconstruction. Thethree-dimensional optical imaging system of imaging system 1900comprises one or more optical cameras 1950, such as an RGB-D camera,configured to generate distance/depth data for a surface of athree-dimensional object, such as a region of interest 55 of a patient50.

The depicted embodiment comprises a single such optical camera 1950.However, as previously mentioned, other numbers and/or types of camerasmay be used. Camera 1950 may be physically decoupled from the x-raytomosynthesis image reconstruction system, as depicted in FIG. 19. Thus,for example, camera 1950 may be mounted to a stand, table, or otherexternal component of the system.

Camera 1950 may be used to observe and/or track various items, such as asurgical tool 20 and/or patient 50. Because camera 1950 is preferablyregistered to a three-dimensional image being reconstructed by the x-raytomosynthesis image reconstruction system, a current trajectory 1966 ofthe surgical tool 20 can be generated and, in some embodiments andimplementations, may be reproduced on a display 1964 in thereconstructed region of interest 55 along with one or moreelements/features in the reconstructed region of interest 55. In somesuch embodiments and implementations, system 1900 may be configured togenerate and/or display trajectory 1966 before tool 20 has enteredregion of interest 55 based upon its tracked movement outside of patient50. This may be useful for intra-operative planning. For example, thisfeature may allow a surgeon/technician to select a desired skin entrypoint and “navigate” to a target point, allowing the surgeon to adjustduring the tool insertion.

In some such embodiments and implementations, system 1900 may beconfigured to generate and/or display other elements used to assist asurgeon/technician during a procedure. For example, as also shown inFIG. 19, a user may be allowed to input a target 1967 and this targetmay be displayed along with the reconstructed image on display/monitor1964. By comparing target 1967 with the current trajectory 1966 of tool20, system 1900 may be configured to allow a surgeon/technician or otheruser to select a preferred trajectory for the surgical instrumentrelative to the region of interest and/or dynamically calculate avariance metric between the preferred trajectory and the currenttrajectory 1966. This may allow, for example, system 1900 to createand/or display a correctional trajectory 1968 to target 1967 so that auser can adjust movement of tool 20 in real time, or near real time,during surgery.

In some embodiments and implementations, system 1900 may be configuredsuch that the display 1934 displays other information, such as a numbercorresponding with the variance metric, in addition to or as analternative to the image in FIG. 19 illustrating both the trajectory1966 and the preferred trajectory 1968. In some embodiments andimplementations, system 1900 may be configured such that a user canselect a target within the region of interest of a target object, suchas target 1967, and system 1900 may also display a current distancebetween the tool 20 and the target 1967.

In some embodiments and implementations, system 1900 may be configuredto dynamically adjust the region of interest 55 in response to movementof the tool 20. For example, in some such embodiments andimplementations, system 1900 may be configured to dynamically define theregion of interest 55 so as to contain a point adjacent to a distal tipof the tool 20, such as dynamically modifying display 1964 as the regionof interest 55 is defined by movement of the distal tip of the tool 20.

FIG. 20 depicts a schematic representation of still another example ofan imaging system 2000. As depicted in this figure, a 3D optical system2050, which may comprise an infrared tracking system or, in otherembodiments, another suitable tracking system, may be combined with oneor more x-ray systems 2020 to provide a 4D reconstruction. In someembodiments and implementations, the 4D reconstruction may comprise amodel-based reconstruction. In the depicted embodiments, two x-raysystems 2020A and 2020B are depicted, although those of ordinary skillin the art will appreciate that the same optical system may be used toperform the steps of systems 2020A and 2020B.

As depicted in FIG. 20, the optical tracking system 2050 may provide afirst 3D surface reconstruction at t=t₀ when the imaged object is in aninitial Position and Orientation (PnO) PnO_0 at 2052. The opticaltracking system 2050 may then provide a second 3D surface reconstructionat t=t₁ when the imaged object is in Position and Orientation (PnO)PnO_1 at 2054, which differs from PnO_0.

The motion between PnO_0 and PnO_1 may then be estimated at 2056. Forexample, in some embodiments and implementations, the motion of the 3Dsurface may be assumed to be rigid, or at least substantially rigid, andthe motion may be identified that minimizes the difference between thetwo surfaces (i.e., PnO_1-0). This may represent the translation andseries of rotations that explain the motion of the 3D surface. Inembodiments and implementations utilizing a tracking system, however,there may no need to infer the motion from the surface. Instead, thesystem may directly provide the motion from, for example, reflectivefiducial markers, such as markers 1651-1653 in system 1600.

One or more x-ray systems, such as x-ray system 2020A, may also providex-ray projections of the imaged object, or at least a portion of theimaged object, preferably at or at least substantially at t=t₀, asindicated at step 2022. Preferably, the x-ray system 2020A is used toreconstruct a 3D image of the object via tomographic reconstruction. Ineven more preferred embodiments and implementations, the x-ray system2020A provides tomosynthesis reconstruction of at least a portion of theimaged object. The 3D image may represent the object at t₀ in PnO_0.

Motion correction may then be applied to the reconstructions from boththe optical system 2050 and the x-ray system 2020A at 2058. For example,in some embodiments and implementations, a 3D to 2D image registrationalgorithm may be used based on the projection(s) of the 3D image byminimizing the difference with the actual measured projections.

The 3D image may be updated by applying translation and/or rotation(“virtually moving”) from the optical system 2050 using the motionestimation PnO_1-0. This image may represent the object at t=t₀ inposition PnO_1.

X-ray system 2020B (or x-ray system 2020A) may then provide projectionsof the imaged object (or a portion of the imaged object) at a time at orat least substantially equal to time t₁ (while the object is in PnO_1).The projection at or at least substantially at t=t₁, together with thevirtually-moved 3D image, may be used in a model-based reconstruction,as discussed above, at step 2060 to provide a 3D image of the object att=t₁, in PnO_1 at 2070.

In some embodiments and implementations, the newly used projections canbe used to model discrepancies between the observed projections and theprojections of the virtually moved object. For example, if a surgicaltool was added between t₁ and t₀, system 2000 may be able to reconstructthe surgical tool by subtracting the newly acquired projections withvirtual projections from the virtually moved model, as explained above.

Other aspects of various embodiments and/or implementations in themotion compensation area may involve combining, or blending (forexample, by averaging or otherwise taking into account) or by usingdetected motion as a seed for an optimization engine that attempts toobserve such motion for later use in a 4D mass attenuationreconstruction. In embodiments using a 3D optical reconstruction system,such as system 2000, motion can be estimated based on patient surfacedisplacements.

As previously mentioned, model-based or layered-based 4D reconstructionmay utilize tracking systems, such as 3D tracking systems in someembodiments. A mass density 4D reconstruction system may model the sceneby assuming that the imaged object is a composition of domains, forexample, objects or layers, that may themselves be modeled and/orreconstructed individually and then can be recombined into a global 4Dreconstructed scene. Each domain model or reconstruction may benefit viaone or more of the mechanisms described above. One such technicalmechanism can be the identification of domain projection matrices fromthe tracking system, when the tracking system is capable to track thedomains individually. For example, an optical tracking system can trackmultiple surgical tools and the patient by having individual opticalreferences, and an optical 3D reconstruction system can track multipleobjects via segmentation and modeling based on rigid object/color andmotion.

A more specific example of such a system is shown in FIG. 21 at 2100.System 2100 may comprise one or more x-ray systems, such as x-raytomosynthesis image reconstruction systems, and one or more trackingsystems, such as 3D tracking systems. Although two x-ray systems 2120Aand 2120B and two tracking systems 2150A and 2150B are depicted in FIG.21, it should be understood that a single x-ray system and a singletracking system may be used instead.

In some embodiments and implementations, system 2100 may be configuredto use motion estimations from both the x-ray system(s) 2120A/2120B andthe tracking system(s) 2150A/2150B by combining motion estimations fromboth modalities.

The x-ray system(s) 2120A/2120B may provide projections of an imagedobject (in PnO_0) at, or at least substantially at, t=t₀. Theprojections may then be used to tomographically reconstruct the objectand provide a 3D image of the object (t=t₀, PnO_0), as indicated at step2122.

Tracking system(s) 2150A/2150B may be used to observe the position ofthe imaged object (in PnO_0) at t=t0.

Updated x-ray projections of the object from x-ray system(s) 2120A/2120Bat substantially t=t₁ may be used to infer the motion of the objectbetween t₀ and t₁, for example, by finding the motion that minimizes thedifference between the new projection and virtual projections of thefirst image. This first motion estimation is indicated at step 2152A.

Tracking system(s) 2150A/2150B may be used to observe the position ofthe object at t=t₁, and infer the motion of the object between t₀ and t₁and provide a second motion estimation at step 2152B

The first and second motion estimations from steps 2152A and 2152B maythen be used in combination to obtain a third, more accurate, motionestimation at 2152C. In some embodiments and implementations, this maybe performed using a weighted average. Alternatively, the second motionestimation may be used as a seed to the first motion estimation, whichmay speed up the first motion estimation.

The projection at t=t₁, together with the initial 3D image and themotion estimation, may be used to provide a 3D reconstruction 2160 ofthe object at t=t₁, in PnO_1. For example, the initial 3D image can befirst virtually moved given the estimated motion and, together with thenewly acquired projections, can be used in a model based reconstruction,as previously mentioned. This image is more accurate than the initialreconstruction 2122 because the newly-acquired projections can be usedto model discrepancies between the observed projections and theprojections of the virtually moved object. For example, if a surgicaltool was added between t₁ and t₀, the system may be configured toreconstruct the surgical tool by subtracting the newly acquiredprojections with virtual projections from the virtually moved model, asexplained above.

As previously mentioned, in some embodiments and implementations, atrajectory of a surgical tool, implant, and/or other movable objectrelative to a region of interest may be visualized in a mass densityreconstruction. For example, a mass density 3D reconstruction may beregistered with a 3D optical image reconstruction system to allow forperforming intraoperative planning when the instrument/implant is stilloutside of the x-ray reconstructed volume (the patient's body, forexample).

In its simplest form, this may be achieved by visualizing the extendedtrajectory of the instrument/implant that is still outside the body,entering the body, and/or inside the body and accounting for itstrajectory and/or entry point. This may allow for reducing x-ray doses,as shots are not needed to visualize the instrument and it is insteadvisualized through the optical system.

As another example for a possible use for this technology, in someembodiments and implementations, a primary axis of the tool/implant maybe extrapolated into an x-ray volume and may define the targeteddirection with respect to anatomical organs and previously embedded, ifany, surgical hardware. As another example, if the destination point hasbeen already selected (e.g., the target 1967 from system 1900), thepractitioner/user can be provided with an estimation of the distancebetween the actual position of the tip and the planned position of thetip, measured along the tool axis.

In some embodiments and implementations, one or more images collectedfrom a tracking system, such as a 3D optical image reconstructionsystem, may be used in a smart user interface to disambiguate the actualtool/implant of interest vs. any other instrumentation/objects that maybe in the image. For example, by identifying the tool/implant that is ina surgeon's hand, a mass attenuation volumetric reconstruction can bemore accurately re-sliced based on analysis of the mass attenuationreconstruction. For example, single value decomposition can be used toidentify a long tool axis or another suitable axis of a movable objectrelative to the region of interest.

Knowledge of the instrument/tool/implant position and its extendedtrajectory from a tracking system may also be used to define a localregion of interest for a mass attenuation reconstruction system. Thismay allow the system to reconstruct a volume that can be centered aroundthe instrument and therefore can be smaller, have higher resolution,and/or exclude extraneous objects. This may reduce the reconstructiontime and may improve the reconstruction resolution. Another benefit ofsome implementations of this method is that it may make thereconstruction more robust by excluding other objects. This may beespecially true when reconstructing a specific data layer, such as theinstrument or implant layer in a reconstruction algorithm. The localregion of interest can also be of a different dimension. For example, a2D slice that is linked to the instrument/tool/implant geometry may betaken, which may further speed up reconstruction as a 2D reconstructionis orders of magnitude faster than 3D reconstruction.

As another example of an instrument/tool reconstruction improvement,data from a tracking system may be used to constrain the reconstructionof a 3D x-ray image system by providing additional information about thereconstructed object, such as geometric information (diameter, length,etc.), material composition (for example by associating colors andreflection or opacity to, for example, different instrument/implantdensities), and/or other information. Such information can be used toconstrain the instrument/implant reconstruction from the x-ray system.Indeed, preliminary data generated by the applicant shows that a densityconstraint may strongly improve reconstruction. The automatic visualidentification of an instrument/implant may be done by identifyingcertain attributes (for example, a color and/or sizes given a surgicaltoolkit from a given manufacturer) into a limited space ofpossibilities, such as the possible values within a specific surgicaltool set being used. This may add an additional dimension of informationon the correct width, length, or other parameters of theinstrument/implant in the case of geometric information. Alternatively,or additionally, this may allow the reconstruction to use a specificdensity reconstruction constraint for the instrument/implantreconstruction layer, thereby improving image quality. These constraintscan be statistical constraints as well as hard constraints.

In another example of a combined 3D x-ray image reconstruction systemand 3D optical image reconstruction system, the combined system mayallow both the 3D tracking system and a mass attenuation 3Dreconstruction system to reconstruct with a common framework (orreducible to a common framework) that may be, a priori, known based on ajoint calibration step. This common framework can be used to do all ofthe above (e.g., aid the reconstruction, account for motion, and/orguide the surgery, etc.) in a simple way without requiring a separateregistration step, which could otherwise be intraoperative or real time,which in the surgical case, for example, may impair the surgicalworkflow. The joint calibration step may, in some implementations, bemost accurately achieved by using a common x-ray geometry calibrationfixture (for example, a helix x-ray calibration phantom, or the “cone”calibration phantom used by some tomosynthesis systems), where the samemarkers may be visible in both the x-ray system and the optical system(for example, bbs on a plexiglass structure). Alternatively, the fixturecould have separate markers at fixed and known relative positions in thefixture.

Yet another more specific example of a combined x-ray imaging andtracking system is shown in FIG. 22 at 2200. System 2200 comprises x-raysystem 2220, which may comprise an x-ray tomosynthesis imagereconstruction system, and tracking system 2250, which may comprise a 3Dtracking system. As previously mentioned, this combined x-ray andtracking system 2200 may be used to combine data from systems 2220 and2250 to perform a more accurate 3D reconstruction of at least a portionof an object. For example, in some embodiments and implementations,system 2200 may use tracking system 2250 to generate a densityconstraint, as indicated at 2252, based on a specific tracked tool orimplant and/or the position of the tool/implant, that is registered tothe image space.

In some embodiments and implementations, the constraint may be appliedas a modification of an intermediary solution in an iterativereconstruction algorithm scheme. For example, the densities where thetool is supposed to be may be biased towards an a-priori known tooldensity or, as another example, the densities may be forced towards alower density (in the human tissue range vs. stainless steel if the toolis stainless steel, for example) if there is no tool being used.

In some embodiments and implementations, tracking system 2250 mayprovide tool/implant identification (and therefore an a-priori of thetool/implant density) and the tool/implant PnO may be registered to thex-ray imaging system 2220. The tool/implant identification and the PnOmay then be used to build a density constraint at 2252 (for example, atransfer function that forces densities closer to the tool in the areaswhere the tool is and/or forces lower densities in areas where there isno tool). The x-ray system 2220 may also provide projections of at leasttwo imaged objects (for example, a surgical tool and a portion ofpatient anatomy).

An iterative reconstruction may be performed at 2260 to generate a 3Dimage. In some embodiments and implementations, an algorithm with aconstraint term such as an iterative reconstruction algorithm withregularization, penalization or other constraint, that uses anidentified constraint and/or projections may be used to provide an imageof the combined objects.

It will be understood by those having skill in the art that changes maybe made to the details of the above-described embodiments withoutdeparting from the underlying principles presented herein. For example,any suitable combination of various embodiments, or the featuresthereof, is contemplated.

In any methods disclosed herein comprising one or more steps or actionsfor performing the described method, the method steps and/or actions maybe interchanged with one another. In other words, unless a specificorder of steps or actions is required for proper operation of theembodiment, the order and/or use of specific steps and/or actions may bemodified.

Throughout this specification, any reference to “one embodiment,” “anembodiment,” or “the embodiment” means that a particular feature,structure, or characteristic described in connection with thatembodiment is included in at least one embodiment. Thus, the quotedphrases, or variations thereof, as recited throughout this specificationare not necessarily all referring to the same embodiment.

Similarly, it should be appreciated that in the above description ofembodiments, various features are sometimes grouped together in a singleembodiment, figure, or description thereof for the purpose ofstreamlining the disclosure. This method of disclosure, however, is notto be interpreted as reflecting an intention that any claim require morefeatures than those expressly recited in that claim. Rather, inventiveaspects lie in a combination of fewer than all features of any singleforegoing disclosed embodiment.

Those having skill in the art will appreciate that many changes may bemade to the details of the above-described embodiments without departingfrom the underlying principles of the invention. The scope of thepresent invention should, therefore, be determined only by the followingclaims.

1. An imaging system, comprising: an x-ray tomosynthesis imagereconstruction system configured to generate three-dimensional imagedata of at least an internal portion of a target object under a surfaceof the target object; a three-dimensional optical imaging systemconfigured to reconstruct an image of at least a portion of the surfaceof the target object by generating surface three-dimensional image data,wherein the three-dimensional optical imaging system is registered tothe x-ray tomosynthesis image reconstruction system; and a processorconfigured to apply an image reconstruction algorithm to generate areconstructed three-dimensional image of the target object, wherein thereconstruction algorithm is configured to use the three-dimensionalimage data from the x-ray tomosynthesis image reconstruction system andto use surface three-dimensional image data from the three-dimensionaloptical imaging system as a constraint to improve image quality of thethree-dimensional image data and reconstruct an image of the targetobject.
 2. The imaging system of claim 1, wherein the reconstructionalgorithm comprises an iterative reconstruction technique.
 3. Theimaging system of claim 1, wherein the constraint comprises a densityconstraint.
 4. The imaging system of claim 1, wherein the constraintcomprises a geometric constraint.
 5. The imaging system of claim 3,wherein the three-dimensional optical imaging system is furtherconfigured to reconstruct an image of at least a portion of a surface ofa surgical instrument or implant by generating surface three-dimensionalimage data for the at least a portion of the surface of the surgicalinstrument or implant, wherein the density constraint comprises at leastin part a density profile derived from the surgical instrument orimplant, and wherein the reconstruction algorithm is configured to applythe density profile of the surgical instrument or implant as aconstraint to improve image quality of the three-dimensional image data.6. The imaging system of claim 3, wherein the system is configured toapply a constraint of zero density from the surface three-dimensionalimage data.
 7. The imaging system of claim 6, wherein the target objectcomprises a patient, and wherein the constraint of zero density isapplied to a region outside of the at least a portion of the surface ofthe target object and outside of at least a portion of a surface of asurgical instrument.
 8. The imaging system of claim 1, wherein at leasta portion of the constraint is derived from an a priori,three-dimensional mass attenuation image registered to the at least aportion of the surface of the target object via surface registration. 9.An imaging system, comprising: an x-ray tomosynthesis imagereconstruction system configured to generate three-dimensional imagedata of a region of interest of a target object under a surface of thetarget object; a three-dimensional optical imaging system configured togenerate surface three-dimensional image data of at least a portion ofthe target object, wherein the optical imaging system is registered tothe x-ray tomosynthesis image reconstruction system, wherein thethree-dimensional optical imaging system is further configured togenerate surface three-dimensional image data of a tool to be insertedinto the region of interest of the target object, and wherein thethree-dimensional optical imaging system is further configured togenerate surface three-dimensional image data of the tool as the toolmoves relative to the surface of the target object; a processorconfigured to compile the surface three-dimensional image data of thetool over time and derive a trajectory of the tool relative to thetarget object; and a display configured to display at least a portion ofthe region of interest and to dynamically display a trajectory of thetool relative to the region of interest.
 10. The imaging system of claim9, wherein the tool comprises a surgical instrument.
 11. The imagingsystem of claim 10, wherein the system is configured to allow a user toselect a preferred trajectory for the surgical instrument relative tothe region of interest, and wherein the processor is configured todynamically calculate a variance metric between the preferred trajectoryand the trajectory.
 12. The imaging system of claim 11, wherein thedisplay is configured to display at least one of a number correspondingwith the variance metric and an image illustrating both the trajectoryand the preferred trajectory.
 13. The imaging system of claim 9, whereinthe system is configured to allow a user to select a target within theregion of interest of the target object, and wherein the system isconfigured to dynamically display a distance between the tool and thetarget.
 14. The imaging system of claim 9, wherein the imaging system isconfigured to dynamically adjust the region of interest in response tomovement of the tool.
 15. The imaging system of claim 14, wherein theimaging system is configured to dynamically define the region ofinterest so as to contain a point adjacent to a distal tip of the tool.16. The imaging system of claim 15, wherein the imaging system isconfigured to dynamically modify the display as the region of interestis defined by movement of the distal tip of the tool.
 17. Afour-dimensional imaging system, comprising: an x-ray tomosynthesisimage reconstruction system configured to generate three-dimensionalimage data of at least a portion of a target object; a tracking systemconfigured to track movement of the at least a portion of the targetobject and generate motion data for a motion model based upon movementof the at least a portion of the target object; and a processorconfigured to generate a reconstructed three-dimensional image of the atleast a portion of the target object over time comprisingfour-dimensional image data, wherein the reconstruction algorithm isconfigured to use the three-dimensional image data from the x-raytomosynthesis image reconstruction system and to use motion data fromthe tracking system to generate the four-dimensional image data.
 18. Thefour-dimensional imaging system of claim 17, wherein the motion modelcomprises use of a rigid transformation.
 19. The four-dimensionalimaging system of claim 17, wherein the tracking system comprises athree-dimensional tracking system.
 20. The four-dimensional imagingsystem of claim 17, wherein the tracking system comprises athree-dimensional imaging system, and wherein the three-dimensionalimaging system is configured to use the motion data from thethree-dimensional imaging system to generate movement of thereconstructed three-dimensional image.
 21. The four-dimensional imagingsystem of claim 17, wherein the x-ray tomosynthesis image reconstructionsystem is further configured to generate motion data based upon movementof the at least a portion of the target object, and wherein the imagingsystem is configured to combine the motion data from the tracking systemwith the motion data from the x-ray tomosynthesis image reconstructionsystem to generate movement of the reconstructed three-dimensionalimage.
 22. An imaging system, comprising: a three-dimensional trackingsystem configured to generate a first data layer comprising motion dataof a tool or implant in motion with respect to a target object; an x-raytomosynthesis imaging system configured to obtain projective image dataof at least a portion of the target object and the tool or implant inmotion with respect to the target object, wherein the three-dimensionaltracking system is registered to the x-ray tomosynthesis imaging system;and a processor configured to generate a second data layer from thethree-dimensional tracking system and from the projective image datafrom the x-ray tomosynthesis imaging system, wherein the processor isconfigured to use a reconstruction algorithm to reconstruct the firstdata layer and the second data layer individually, wherein the firstdata layer and the second data layer have different constraints, andwherein the processor is configured to combine the first data layer withthe second data layer to generate a reconstructed three-dimensionalimage of at least a portion of the target object with the tool orimplant.
 23. The imaging system of claim 22, wherein thethree-dimensional tracking system is configured to identify the tool orimplant using an a priori density profile, and wherein thethree-dimensional tracking system is further configured to use a deriveddensity profile based on the tool or implant to improve thereconstruction of the second data layer and thereby improve thereconstruction of the three-dimensional image.
 24. The imaging system ofclaim 22, wherein the three-dimensional tracking system comprises athree-dimensional optical imaging system configured to generate themotion data by tracking movement of the tool or implant.
 25. The imagingsystem of claim 22, wherein at least one of a shape and a color of thetool or implant is used to identify the tool or implant using an apriori density profile and a derived density profile based on the toolor implant to improve the reconstruction of the second data layer andthereby improve the reconstruction of the three-dimensional image.